MIN-Entropy: A New Signature File Declustering Algorithm for Intra-Query Parallelism

Intra-query parallelism is important for achieving high performance in parallel processing environment. For processing a signatrue file in parallel, an effective declustering algon’thm that must avoid data skew and execution skew is needed. The Linear Code Decomposition Method(LCDM) that is used for the Hamming filter may give a good performance, but it fails to evenly decluster a signature file when data is skewed. In addition, it has other problems such as limited scalability and non-determinism. In this paper we propose a new signature file declustering algorithm, called MIN-entropy, that overcomes those problems in the LCDM. The MIN-entropy declusters a signature file based on a measure of execution load uniformity, called signature entropy, that is derived from the previously declustered signatures. Since the MIN-entropy uses dynamic information such as accumulated signatures, it provides high performance for a variety of workloads and configurations. We show through the simulation experiments that the MIN-entropy outperforms the LCDM under various data distributions.